

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>增加/删除 OSD &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/css/custom.css" type="text/css" />

  
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
        <script src="../../../_static/jquery.js"></script>
        <script src="../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="../../../" id="documentation_options" src="../../../_static/documentation_options.js"></script>
        <script src="../../../_static/doctools.js"></script>
        <script src="../../../_static/sphinx_highlight.js"></script>
    
    <script type="text/javascript" src="../../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../../genindex/" />
    <link rel="search" title="Search" href="../../../search/" />
    <link rel="next" title="增加/删除监视器" href="../add-or-rm-mons/" />
    <link rel="prev" title="Configuring Monitor Election Strategies" href="../change-mon-elections/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../../" class="icon icon-home" aria-label="Home"></a></li>
          <li class="breadcrumb-item"><a href="../../">Ceph 存储集群</a></li>
          <li class="breadcrumb-item"><a href="../">集群运维</a></li>
      <li class="breadcrumb-item active">增加/删除 OSD</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../../_sources/rados/operations/add-or-rm-osds.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../../" class="icon icon-home"> Ceph
          

          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../../start/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../cephadm/">Cephadm</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../../">Ceph 存储集群</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../../configuration/">配置</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../">运维</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../operating/">操纵集群</a></li>
<li class="toctree-l3"><a class="reference internal" href="../health-checks/">健康检查</a></li>
<li class="toctree-l3"><a class="reference internal" href="../monitoring/">监控集群</a></li>
<li class="toctree-l3"><a class="reference internal" href="../monitoring-osd-pg/">监控 OSD 和归置组</a></li>
<li class="toctree-l3"><a class="reference internal" href="../user-management/">用户管理</a></li>
<li class="toctree-l3"><a class="reference internal" href="../pgcalc/">PG Calc</a></li>
<li class="toctree-l3"><a class="reference internal" href="../data-placement/">数据归置概览</a></li>
<li class="toctree-l3"><a class="reference internal" href="../pools/">存储池</a></li>
<li class="toctree-l3"><a class="reference internal" href="../erasure-code/">纠删码</a></li>
<li class="toctree-l3"><a class="reference internal" href="../cache-tiering/">分级缓存</a></li>
<li class="toctree-l3"><a class="reference internal" href="../placement-groups/">归置组</a></li>
<li class="toctree-l3"><a class="reference internal" href="../upmap/">使用 pg-upmap</a></li>
<li class="toctree-l3"><a class="reference internal" href="../read-balancer/">Operating the Read (Primary) Balancer</a></li>
<li class="toctree-l3"><a class="reference internal" href="../balancer/">均衡器模块</a></li>
<li class="toctree-l3"><a class="reference internal" href="../crush-map/">CRUSH 图</a></li>
<li class="toctree-l3"><a class="reference internal" href="../crush-map-edits/">手动编辑一个 CRUSH 图</a></li>
<li class="toctree-l3"><a class="reference internal" href="../stretch-mode/">Stretch Clusters</a></li>
<li class="toctree-l3"><a class="reference internal" href="../change-mon-elections/">Configuring Monitor Election Strategies</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">增加/删除 OSD</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#id1">增加 OSD</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id13">删除 OSD （手动）</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="../add-or-rm-mons/">增加/删除监视器</a></li>
<li class="toctree-l3"><a class="reference internal" href="../devices/">设备管理</a></li>
<li class="toctree-l3"><a class="reference internal" href="../bluestore-migration/">迁移到 BlueStore</a></li>
<li class="toctree-l3"><a class="reference internal" href="../control/">命令参考</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/community/">Ceph 社区</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/troubleshooting-mon/">监视器故障排除</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/troubleshooting-osd/">OSD 故障排除</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/troubleshooting-pg/">归置组排障</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/log-and-debug/">日志记录和调试</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/cpu-profiling/">CPU 剖析</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/memory-profiling/">内存剖析</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../man/">    手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../troubleshooting/">故障排除</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../api/">APIs</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../monitoring/">监控概览</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../hardware-monitoring/">硬件监控</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <section id="osd">
<h1>增加/删除 OSD<a class="headerlink" href="#osd" title="Permalink to this heading"></a></h1>
<p>如果您的集群已经在运行，你可以在运行时添加或删除 OSD 。</p>
<section id="id1">
<h2>增加 OSD<a class="headerlink" href="#id1" title="Permalink to this heading"></a></h2>
<p>可以给集群增加 OSD ，以扩展集群的容量和恢复能力。
通常，一个 OSD 就是一个 <code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code> 守护进程，
它运行在一台主机内的硬盘之上。如果你的机器有多个硬盘，
可以给每个硬盘启动一个 <code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code> 守护进程。</p>
<p>通常，你应该监控集群容量，看是否达到了容量上限。
如果达到了它的 <code class="docutils literal notranslate"><span class="pre">near</span> <span class="pre">full</span></code> 比率，应该增加 OSD 来扩容。</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>不要等集群空间达到 <code class="docutils literal notranslate"><span class="pre">full</span> <span class="pre">ratio</span></code> 再增加 OSD 。
集群空间使用率达到 <code class="docutils literal notranslate"><span class="pre">near</span> <span class="pre">full</span> <span class="pre">ratio</span></code> 比率后，
此后发生的 OSD 故障会导致集群空间超过其 <code class="docutils literal notranslate"><span class="pre">full</span> <span class="pre">ratio</span></code> 。</p>
</div>
<section id="id2">
<h3>部署硬件<a class="headerlink" href="#id2" title="Permalink to this heading"></a></h3>
<p>如果你通过增加主机来增加 OSD ，关于 OSD 服务器硬件的配置请参见<a class="reference external" href="../../../start/hardware-recommendations">硬件推荐</a>。要把一台 OSD 主机加入到集群，首先要安装最新版的 Linux ，而且存储硬盘要做好必要的准备，详情参见<a class="reference external" href="../../configuration/filesystem-recommendations">文件系统推荐</a>。</p>
<p>把 OSD 主机安装到集群机架上，连接好网络、确保网络通畅。详情见<a class="reference external" href="../../configuration/network-config-ref">网络配置参考</a>。</p>
</section>
<section id="id6">
<h3>安装必要软件<a class="headerlink" href="#id6" title="Permalink to this heading"></a></h3>
<p>在手动部署的集群里，你必须手动安装 Ceph 软件包，详情见<a class="reference external" href="../../../install">安装 Ceph （手动）</a>。你应该配置一个无密码登录 SSH 的用户，且他有 root 权限。</p>
</section>
<section id="id7">
<h3>增加 OSD （手动）<a class="headerlink" href="#id7" title="Permalink to this heading"></a></h3>
<p>此过程要设置一个 <code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code> 守护进程，让它使用一个硬盘，
且让集群把数据分布到这个 OSD 。
如果一台主机有多个硬盘，可以重复此过程，
把每个硬盘配置为一个 OSD 。</p>
<p>按照下列步骤的演示，添加 OSD 要依次创建元数据目录、
配置数据存储驱动器、把 OSD 加入集群、
然后把它加入 CRUSH 图。</p>
<p>把这个 OSD 加入 CRUSH 图时，需要仔细考虑给新 OSD 分配的权重。
因为单个存储驱动器的容量随时间增长，
较新的 OSD 主机与集群里的旧主机相比，很可能拥有更大的硬盘，
因此可以配置更大的权重。</p>
<div class="admonition tip">
<p class="admonition-title">Tip</p>
<p>Ceph 在存储池硬件统一的情况下运行得最好，
但是容许增加容量不一的驱动器，并相应地调整它们的权重。
即便如此，为实现最佳性能， CRUSH 的分级结构最好按类型、容量定义。
最好是统一地给现有主机增加更大的驱动器，
此工作可以逐步推进，每次新增驱动器时把较小的驱动器替换掉。</p>
</div>
<ol class="arabic">
<li><p>创建新 OSD ，命令格式如下。
如果操作时未指定 UUID ，
OSD 启动时会自动生成一个。OSD 号，在此命令的输出里有，后续步骤需要用到。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><style type="text/css">
span.prompt1:before {
  content: "$ ";
}
</style><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>create<span class="w"> </span><span class="o">[{</span>uuid<span class="o">}</span><span class="w"> </span><span class="o">[{</span>id<span class="o">}]]</span></span>
</pre></div></div><p>如果指定了可选参数 {id} ，那么它将作为 OSD ID 。
但是，如果此数字已经在用，此命令会出错。</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>不建议明确指定 <code class="docutils literal notranslate"><span class="pre">{id}</span></code> 参数。
因为 ID 是按照数组分配的，跳过一些依然会浪费内存；
尤其是跳过太多、或者集群很大时，这种内存消耗会很明显。
如果不指定 <code class="docutils literal notranslate"><span class="pre">{id}</span></code> ， Ceph 将采用可用的最小 ID 数字，
并且避免这些问题。</p>
</div>
</li>
<li><p>给新 OSD 创建默认目录，执行下列命令：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ssh<span class="w"> </span><span class="o">{</span>new-osd-host<span class="o">}</span></span>
<span class="prompt1">sudo<span class="w"> </span>mkdir<span class="w"> </span>/var/lib/ceph/osd/ceph-<span class="o">{</span>osd-number<span class="o">}</span></span>
</pre></div></div></li>
<li><p>如果准备用于 OSD 的是单独的而非系统盘，要先准备一下。
执行下列命令：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ssh<span class="w"> </span><span class="o">{</span>new-osd-host<span class="o">}</span></span>
<span class="prompt1">sudo<span class="w"> </span>mkfs<span class="w"> </span>-t<span class="w"> </span><span class="o">{</span>fstype<span class="o">}</span><span class="w"> </span>/dev/<span class="o">{</span>drive<span class="o">}</span></span>
<span class="prompt1">sudo<span class="w"> </span>mount<span class="w"> </span>-o<span class="w"> </span>user_xattr<span class="w"> </span>/dev/<span class="o">{</span>hdd<span class="o">}</span><span class="w"> </span>/var/lib/ceph/osd/ceph-<span class="o">{</span>osd-number<span class="o">}</span></span>
</pre></div></div></li>
<li><p>初始化这个 OSD 的数据目录，执行下列命令。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ssh<span class="w"> </span><span class="o">{</span>new-osd-host<span class="o">}</span></span>
<span class="prompt1">ceph-osd<span class="w"> </span>-i<span class="w"> </span><span class="o">{</span>osd-num<span class="o">}</span><span class="w"> </span>--mkfs<span class="w"> </span>--mkkey</span>
</pre></div></div><p>运行 <code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code> 时目录必须是空的。</p>
</li>
<li><p>注册 OSD 认证密钥，执行下列命令：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>auth<span class="w"> </span>add<span class="w"> </span>osd.<span class="o">{</span>osd-num<span class="o">}</span><span class="w"> </span>osd<span class="w"> </span><span class="s1">&#39;allow *&#39;</span><span class="w"> </span>mon<span class="w"> </span><span class="s1">&#39;allow rwx&#39;</span><span class="w"> </span>-i<span class="w"> </span>/var/lib/ceph/osd/ceph-<span class="o">{</span>osd-num<span class="o">}</span>/keyring</span>
</pre></div></div><p>列出的 <code class="docutils literal notranslate"><span class="pre">ceph-{osd-num}</span></code> 路径里的 <code class="docutils literal notranslate"><span class="pre">ceph</span></code> 是集群名字。
如果你的集群名字不是 <code class="docutils literal notranslate"><span class="pre">ceph</span></code> ，
那么 <code class="docutils literal notranslate"><span class="pre">ceph-{osd-num}</span></code> 里的
<code class="docutils literal notranslate"><span class="pre">ceph</span></code> 字符串应该换成你的集群名。例如，
如果你的集群名是 <code class="docutils literal notranslate"><span class="pre">cluster1</span></code> ，那么命令里的路径应该是
<code class="docutils literal notranslate"><span class="pre">/var/lib/ceph/osd/cluster1-{osd-num}/keyring</span></code> 。</p>
</li>
<li><p>把 OSD 加入 CRUSH 图，执行下列命令。这样它才开始收数据。
用 <code class="docutils literal notranslate"><span class="pre">ceph</span> <span class="pre">osd</span> <span class="pre">crush</span> <span class="pre">add</span></code> 命令把 OSD 加入
CRUSH 分级结构的合适位置。如果你指定了不止一个桶，
此命令会把它加入你所指定的桶中最具体的一个，<em>并且</em>把此桶挪到你指定的其它桶之内。
<strong>重要：</strong>如果你只指定了 root 桶，
此命令会把 OSD 直接挂到 root 下面，
但是 CRUSH 规则期望它位于主机内。如果 OSD 不在主机内，
这些 OSD 们可能一点数据也收不到。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>crush<span class="w"> </span>add<span class="w"> </span><span class="o">{</span>id-or-name<span class="o">}</span><span class="w"> </span><span class="o">{</span>weight<span class="o">}</span><span class="w">  </span><span class="o">[{</span>bucket-type<span class="o">}={</span>bucket-name<span class="o">}</span><span class="w"> </span>...<span class="o">]</span></span>
</pre></div></div><p>注意下，还有另外一种方法把新 OSD 加进 CRUSH 图：
反编译 CRUSH 图、把 OSD 加入设备列表、以桶的形式加入它所在的主机
（如果它还没在 CRUSH 图里）、把设备作为 item 加入主机、
给此设备分配权重、重编译 CRUSH 图、并应用它。
详情参见<a class="reference external" href="../crush-map#addosd">增加/移动 OSD</a> 。最近的版本极少需要此方法
（这句话是在 Reef 发布的那个月写下的）。</p>
</li>
</ol>
</section>
<section id="rados-replacing-an-osd">
<span id="id8"></span><h3>替换一个 OSD<a class="headerlink" href="#rados-replacing-an-osd" title="Permalink to this heading"></a></h3>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>如果本节中的步骤对您没用，
试试 <code class="docutils literal notranslate"><span class="pre">cephadm`</span></code> 文档中的说明：
<a class="reference internal" href="../../../cephadm/services/osd/#cephadm-replacing-an-osd"><span class="std std-ref">Replacing an OSD</span></a> 。</p>
</div>
<p>有时需要替换 OSD ：比如磁盘损坏时、
或者管理员想用新后端重新开通 OSD ，比如从
FileStore 切换到 BlueStore 。不像<a class="reference external" href="../crush-map#removeosd">删除 OSD</a> ，
要更换的 OSD ，在经历销毁后，其 ID 和
CRUSH 图条目都需要保持不变。</p>
<ol class="arabic">
<li><p>确保销毁此 OSD 不会有问题：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="k">while</span><span class="w"> </span>!<span class="w"> </span>ceph<span class="w"> </span>osd<span class="w"> </span>safe-to-destroy<span class="w"> </span>osd.<span class="o">{</span>id<span class="o">}</span><span class="w"> </span><span class="p">;</span><span class="w"> </span><span class="k">do</span><span class="w"> </span>sleep<span class="w"> </span><span class="m">10</span><span class="w"> </span><span class="p">;</span><span class="w"> </span><span class="k">done</span></span>
</pre></div></div></li>
<li><p>先销毁这个 OSD:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>destroy<span class="w"> </span><span class="o">{</span>id<span class="o">}</span><span class="w"> </span>--yes-i-really-mean-it</span>
</pre></div></div></li>
<li><p><em>可选</em>：如果这个硬盘不是新的，
之前另作他用，先擦除它：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph-volume<span class="w"> </span>lvm<span class="w"> </span>zap<span class="w"> </span>/dev/sdX</span>
</pre></div></div></li>
<li><p>用销毁前的 OSD ID 来准备这个磁盘：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph-volume<span class="w"> </span>lvm<span class="w"> </span>prepare<span class="w"> </span>--osd-id<span class="w"> </span><span class="o">{</span>id<span class="o">}</span><span class="w"> </span>--data<span class="w"> </span>/dev/sdX</span>
</pre></div></div></li>
<li><p>最后，激活此 OSD ：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph-volume<span class="w"> </span>lvm<span class="w"> </span>activate<span class="w"> </span><span class="o">{</span>id<span class="o">}</span><span class="w"> </span><span class="o">{</span>fsid<span class="o">}</span></span>
</pre></div></div></li>
</ol>
<p>或者，不需要最后两步（准备硬盘、再激活 OSD ），
只需一个命令即可重建此 OSD ，执行下列命令：</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph-volume<span class="w"> </span>lvm<span class="w"> </span>create<span class="w"> </span>--osd-id<span class="w"> </span><span class="o">{</span>id<span class="o">}</span><span class="w"> </span>--data<span class="w"> </span>/dev/sdX</span>
</pre></div></div></div></blockquote>
</section>
<section id="id9">
<h3>启动 OSD<a class="headerlink" href="#id9" title="Permalink to this heading"></a></h3>
<p>把 OSD 加入 Ceph 后， OSD 就在集群里了。然而，在启动前，
它的状态一直是 <code class="docutils literal notranslate"><span class="pre">down</span></code> 且 <code class="docutils literal notranslate"><span class="pre">in</span></code> 。这个 OSD 没在运行，
且不能接收数据。要启动 OSD ，
可以用管理主机上的 <code class="docutils literal notranslate"><span class="pre">service</span> <span class="pre">ceph</span></code> 、或从 OSD 所在主机启动：</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">sudo<span class="w"> </span>systemctl<span class="w"> </span>start<span class="w"> </span>ceph-osd@<span class="o">{</span>osd-num<span class="o">}</span></span>
</pre></div></div></div></blockquote>
<p>一旦你启动了 OSD ，其状态就变成了 <code class="docutils literal notranslate"><span class="pre">up</span></code> 且 <code class="docutils literal notranslate"><span class="pre">in</span></code> 。</p>
</section>
<section id="id10">
<h3>观察数据迁移<a class="headerlink" href="#id10" title="Permalink to this heading"></a></h3>
<p>把新 OSD 加入 CRUSH 图后， Ceph 开始重新均衡集群，一些归置组（ PG ）
会迁移到新 OSD 里，可以用 <a class="reference external" href="../monitoring">ceph</a> 工具观察此过程，执行下列命令：</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>-w</span>
</pre></div></div></div></blockquote>
<p>或者：</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">watch<span class="w"> </span>ceph<span class="w"> </span>status</span>
</pre></div></div></div></blockquote>
<p>你会看到归置组状态从 <code class="docutils literal notranslate"><span class="pre">active+clean</span></code> 变为 <code class="docutils literal notranslate"><span class="pre">active,</span> <span class="pre">some</span> <span class="pre">degraded</span> <span class="pre">objects</span></code>
（有降级的对象）、且迁移完成后回到 <code class="docutils literal notranslate"><span class="pre">active+clean</span></code> 状态。
观察完后，键盘按 Ctrl-C 退出。</p>
</section>
</section>
<section id="id13">
<h2>删除 OSD （手动）<a class="headerlink" href="#id13" title="Permalink to this heading"></a></h2>
<p>可以在集群运行时手动删除 OSD ：在缩减集群尺寸或替换硬件时可以这样操作。
通常，一个 OSD 就是一个 <code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code> 守护进程、
它运行在主机内的一个硬盘之上。
另外，如果一台主机上有多个存储驱动器，你可能得删除多个
<code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code> ：主机上一个守护进程对应一个驱动器。</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>删除 OSD 前，确保集群没接近 <code class="docutils literal notranslate"><span class="pre">full</span> <span class="pre">ratio</span></code> 比率。
否则，删除 OSD 可能导致集群达到或超过 <code class="docutils literal notranslate"><span class="pre">full</span> <span class="pre">ratio</span></code> 值。</p>
</div>
<section id="osd-out">
<h3>把 OSD 踢出集群（ <code class="docutils literal notranslate"><span class="pre">out</span></code> ）<a class="headerlink" href="#osd-out" title="Permalink to this heading"></a></h3>
<p>删除 OSD 前，它通常是 <code class="docutils literal notranslate"><span class="pre">up</span></code> 且 <code class="docutils literal notranslate"><span class="pre">in</span></code> 的。要想从集群删除此 OSD ，
要先把它踢出（ <code class="docutils literal notranslate"><span class="pre">out</span></code> ）集群，以使 Ceph 启动重新均衡、
把数据复制到其他 OSD 。要把 OSD 踢出（ <code class="docutils literal notranslate"><span class="pre">out</span></code> ）集群，
执行下列命令：</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>out<span class="w"> </span><span class="o">{</span>osd-num<span class="o">}</span></span>
</pre></div></div></div></blockquote>
</section>
<section id="id14">
<h3>观察数据迁移<a class="headerlink" href="#id14" title="Permalink to this heading"></a></h3>
<p>Observing the Data Migration</p>
<p>一旦把 OSD 踢出（ <code class="docutils literal notranslate"><span class="pre">out</span></code> ）集群， Ceph 就会开始重新均衡集群、把归置组迁出将删除的 OSD 。可以用 <a class="reference external" href="../monitoring">ceph</a> 工具观察此过程，执行下列命令：</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>-w</span>
</pre></div></div></div></blockquote>
<p>你会看到 PG 状态从 <code class="docutils literal notranslate"><span class="pre">active+clean</span></code> 变为 <code class="docutils literal notranslate"><span class="pre">active,</span> <span class="pre">some</span> <span class="pre">degraded</span> <span class="pre">objects</span></code> 、
迁移完成后最终回到 <code class="docutils literal notranslate"><span class="pre">active+clean</span></code> 状态。
观察完后，键盘按 Ctrl-C 退出。</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>有时候，踢出（ <code class="docutils literal notranslate"><span class="pre">out</span></code> ）某个 OSD
可能会使 CRUSH 进入临界状态，
这时某些 PG 一直卡在 <code class="docutils literal notranslate"><span class="pre">active+remapped</span></code> 状态。这个问题有时发生在只有几台主机的小集群上（比如，小型测试集群）。
要解决这个问题，把此 OSD 标记为 <code class="docutils literal notranslate"><span class="pre">in</span></code> ，执行下列命令：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="o">{</span>osd-num<span class="o">}</span></span>
</pre></div></div><p>等这个 OSD 回到最初的状态后，不要再把它标记为 <code class="docutils literal notranslate"><span class="pre">out</span></code> 。
而是把此 OSD 的权重设置为 <code class="docutils literal notranslate"><span class="pre">0</span></code> ，
执行下列命令：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>crush<span class="w"> </span>reweight<span class="w"> </span>osd.<span class="o">{</span>osd-num<span class="o">}</span><span class="w"> </span><span class="m">0</span></span>
</pre></div></div><p>调整此 OSD 的权重后，观察引发的数据迁移，
确认它成功完成。把某一 OSD 标记为 <code class="docutils literal notranslate"><span class="pre">out</span></code> 和权重改为 <code class="docutils literal notranslate"><span class="pre">0</span></code> 的区别在于对包含此 OSD 的桶做出的操作，
OSD 被标记为 <code class="docutils literal notranslate"><span class="pre">out</span></code> 时，它所在桶的权重没变；
而 OSD 权重改成 <code class="docutils literal notranslate"><span class="pre">0</span></code> 后，它所在桶的权重更新了
（也就是，这个 OSD 的权重从桶的总权重中减掉了）。
操作小型集群时，有时候用上述的 reweight 命令更好。</p>
</div>
</section>
<section id="id15">
<h3>停止 OSD<a class="headerlink" href="#id15" title="Permalink to this heading"></a></h3>
<p>把 OSD 踢出（ <code class="docutils literal notranslate"><span class="pre">out</span></code> ）集群后，它可能仍在运行，
此时，此 OSD 的状态仍然是 <code class="docutils literal notranslate"><span class="pre">up</span></code> 且 <code class="docutils literal notranslate"><span class="pre">out</span></code> 。
从集群删除此 OSD 前，必须先停止这个 OSD 。执行下列命令：</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ssh<span class="w"> </span><span class="o">{</span>osd-host<span class="o">}</span></span>
<span class="prompt1">sudo<span class="w"> </span>systemctl<span class="w"> </span>stop<span class="w"> </span>ceph-osd@<span class="o">{</span>osd-num<span class="o">}</span></span>
</pre></div></div></div></blockquote>
<p>停止 OSD 后，状态变为 <code class="docutils literal notranslate"><span class="pre">down</span></code> 。</p>
</section>
<section id="id16">
<h3>删除 OSD<a class="headerlink" href="#id16" title="Permalink to this heading"></a></h3>
<p>下列步骤把一个 OSD 移出集群运行图、删除此 OSD 的认证密钥、
删除 OSD 运行图条目、删除 <code class="docutils literal notranslate"><span class="pre">ceph.conf</span></code> 条目。
如果主机有多个硬盘，每个硬盘对应的 OSD 都得重复此步骤。</p>
<ol class="arabic">
<li><p>首先，让集群忘掉这个 OSD 。这一步会从 CRUSH 图中删掉这个
OSD 、删除其认证密钥，也会从 OSD 运行图中删掉。
（ <a class="reference internal" href="../../../man/8/ceph/#ceph-admin-osd"><span class="std std-ref">purge 子命令</span></a> 在 Luminous 中才引进，更老的版本看 <a class="reference internal" href="#ceph-osd-purge-procedure-pre-luminous"><span class="std std-ref">这里链接的步骤</span></a> ）：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>purge<span class="w"> </span><span class="o">{</span>id<span class="o">}</span><span class="w"> </span>--yes-i-really-mean-it</span>
</pre></div></div></li>
<li><p>登录保存着 <code class="docutils literal notranslate"><span class="pre">ceph.conf</span></code> 主副本的主机：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ssh<span class="w"> </span><span class="o">{</span>admin-host<span class="o">}</span></span>
<span class="prompt1"><span class="nb">cd</span><span class="w"> </span>/etc/ceph</span>
<span class="prompt1">vim<span class="w"> </span>ceph.conf</span>
</pre></div></div></li>
<li><p>删除 <code class="docutils literal notranslate"><span class="pre">ceph.conf</span></code> 文件内的相关 OSD 条目（如果还有的话）：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">osd</span><span class="mf">.1</span><span class="p">]</span>
        <span class="n">host</span> <span class="o">=</span> <span class="p">{</span><span class="n">hostname</span><span class="p">}</span>
</pre></div>
</div>
</li>
<li><p>在保存着 <code class="docutils literal notranslate"><span class="pre">ceph.conf</span></code> 文件主副本的主机上操作，把更新过的
<code class="docutils literal notranslate"><span class="pre">ceph.conf</span></code> 文件复制到集群内其余主机的 <code class="docutils literal notranslate"><span class="pre">/etc/ceph</span></code> 目录下。</p></li>
</ol>
<p id="ceph-osd-purge-procedure-pre-luminous">如果你的 Ceph 集群版本低于 Luminous ，就不能用
<code class="docutils literal notranslate"><span class="pre">ceph</span> <span class="pre">osd</span> <span class="pre">purge</span></code> 子命令。手动执行如下步骤：</p>
<ol class="arabic">
<li><p>删除 CRUSH 图的对应 OSD 条目，它就不再接收数据了。
详情参见<a class="reference external" href="../crush-map#removeosd">删除 OSD</a> ：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>crush<span class="w"> </span>remove<span class="w"> </span><span class="o">{</span>name<span class="o">}</span></span>
</pre></div></div><p>除了从 CRUSH 图删除此 OSD ，你还有两个选择，可以按其中之一操作：
(1) 反编译 CRUSH 图、删除 device 列表条目、删除对应的 host 桶条目；
(2) 从 CRUSH 图删除这个 host 桶
（如果它在 CRUSH 图里，而且你想删除这个主机），
重编译 CRUSH 图并应用它。</p>
</li>
<li><p>删除 OSD 认证密钥：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>auth<span class="w"> </span>del<span class="w"> </span>osd.<span class="o">{</span>osd-num<span class="o">}</span></span>
</pre></div></div></li>
<li><p>删除 OSD 。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>rm<span class="w"> </span><span class="o">{</span>osd-num<span class="o">}</span></span>
</pre></div></div><p>例如：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>rm<span class="w"> </span><span class="m">1</span></span>
</pre></div></div></li>
</ol>
</section>
</section>
</section>



<div id="support-the-ceph-foundation" class="admonition note">
  <p class="first admonition-title">Brought to you by the Ceph Foundation</p>
  <p class="last">The Ceph Documentation is a community resource funded and hosted by the non-profit <a href="https://ceph.io/en/foundation/">Ceph Foundation</a>. If you would like to support this and our other efforts, please consider <a href="https://ceph.io/en/foundation/join/">joining now</a>.</p>
</div>


           </div>
           
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../change-mon-elections/" class="btn btn-neutral float-left" title="Configuring Monitor Election Strategies" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../add-or-rm-mons/" class="btn btn-neutral float-right" title="增加/删除监视器" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).</p>
  </div>

   

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>